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Abstract 

A video game called ‘Re-Mission ’ has recently been investigated with adolescent and young adult cancer patients 
enrolled in a multi-site randomized controlled evaluation of the game as a psycho-educational intervention. The 
main focus of the trial was to determine effects of the game on self-care and other health-related outcomes. It was 
also considered valuable to evaluate participants’ perceptions of the game as (1) acceptable as a treatment-related 
activity for young cancer patients, and (2) credible as an intervention designed to change patients’ knowledge, 
attitudes and self-care behaviors relating to treatment. Although the cancer-related content of the game is informed 
by surveys of cancer professionals and patients themselves, acceptability and credibility with end-users have been 
important factors influencing the usability and efficacy of a range of psychological interventions. As part of the 
multi-site trial, 197 patients with cancer, between the ages of 13 and 29, were assigned to the treatment group. Most 
patients (148) completed a 9-item acceptability/credibility rating scale following 3-months’ access to Re-Mission. 
These ‘completers ’ played Re-Mission more than the other patients, but did not differ from them on gender, age, or 
prior game experience. Responses to the questionnaire were analyzed as two factors representing acceptability and 
credibility. A mean rating for acceptability (4.1 on scale of 5) indicated a good level of acceptability, and mean 
rating for credibility (3.7 on scale of 5) indicated a moderate level of belief in the game as an effective intervention. 
Correlation analyses showed that whereas acceptability and credibility ratings were not significantly associated 
with educational level, both were significantly (p < 0.01 ) and positively correlated with amount of time spent 
playing the video game during treatment (acceptability: r = 0.26, credibility: r = 0.25 ). The findings indicate that 
the self-care intervention video game would be a useful addition to the psycho-educational resources available to 
treatment teams. 
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Introduction 

There have been several published reports describing 
the use of videogames as psycho-educational 
interventions for young people being treated for 
chronic illnesses, including asthma, diabetes and 
cancer. 1-3 The educational content of these games 
has been designed to increase illness-related 
knowledge and change attitudes and behaviors 
related to treatment adherence, based on the hope that 
these changes will lead to better health outcomes. 1 
These published reports generally focused on 
analyses of effects of videogame use on relevant 
behavioral variables, including knowledge about 
illness and treatment, and adherence to prescribed 
treatment regimens. 1 ' 2 Less attention has been paid 
to the measurement or analysis of users’ perceptions 
about the purpose or likely effects of the game they 
have been asked to play, however a few studies have 
elicited players’ views about how much they enjoyed 
using the game or whether they would recommend it 
to a friend. 4 ' 7 Also, previous studies of the use of 
health videogames have not collected accurate data 
on the amount of time users spent playing the 
videogame, although in one instance this was 
estimated from players’ recollections. 7 

The lack of published data on patients’ perceptions 
about health videogames, or their usage of these, may 
represent an important gap in knowledge about the 
potential value of health videogames as adjuvant 
psycho-educational interventions for chronic 
illnesses. Although there is evidence that health 
videogames could well play an important role in 
helping young people through treatment for chronic 
illnesses, it is unclear whether young patients regard 
health videogames as serious interventions which 
could help achieve treatment goals. Specifically, 
there has been no exploration of whether patients’ 
use of health videogames is systematically related to 
their perceptions of the acceptability, credibility or 
efficacy of the videogame. The importance of this 
issue stems from the illness perception literature 
showing that patients’ perceptions of interventions 
play a critical role in how those interventions impact 
the primary health outcomes, to the extent that 
patients’ beliefs about some standard interventions 
are at least as important as both the intervention itself 
and adherence to it, in determining treatment 
outcomes. 8 ’ 11 

Reviews indicate that published studies of the 
efficacy of health videogames generally have not 
reported formal data on how patients actually used 
the videogame provided, although they generally 


were given some instruction about use (e.g., how 
often or for how long). 1 ' 2 Yet such data on usage can 
lead to a better understanding of the intervention. 
Where instructions are given, they represent a 
treatment regimen and therefore provide an 
opportunity to measure treatment adherence. Even 
where the treatment is just an adjunct to some 
primary therapy, as often is the case with psycho- 
educational therapies, adherence is still important. A 
measure of adherence might be helpful in 
understanding why some patients respond more than 
others to the videogame intervention. It would also 
allow exploration of possible associations between 
adherence and patients’ perceptions of the value of 
the intervention. Finally, it provides a basic measure 
of treatment integrity which should be an essential 
component of any intervention evaluation. 12 

The research described in this report explores the 
self-reported perceptions of young users of a health- 
oriented videogame about the credibility and 
acceptability of the videogame used as an adjunct to 
standard treatment for cancer. The research 
investigates possible associations between patients’ 
perceptions of the game and their use of the game 
during treatment, and whether game use or 
perceptions are related to patients’ educational level 
or ethnicity. 

The videogame ‘Re-Mission’ evaluated in this study 
is played on a mini-PC using commercial game-pad 
controllers. It is an action game in which the player 
can manipulate a humanoid robot character (nanobot ) 
named “Roxxi” inside the 3-D virtual bodies of 19 
different young patients with seven cancers common 
in the target population (e.g., ALL, AML, Hodgkin’s 
disease, non-Hodgkin’s lymphoma, osteosarcoma, 
Ewing’s sarcoma, and brain tumors). “Roxxi” is 
accompanied by “Smitty,” her mentor who delivers 
educational narrative over game play. The game 
consists of three tutorial missions, two 2-player 
missions and 20 single-player missions. The player 
can choose to play the game in English, Spanish, or 
French. Players win by destroying cancer cells and 
other enemies in the body (e.g., bacteria, mucositis 
lesions, stool jags) but need to be careful not to waste 
ammunition or cause secondary damage to the body. 
Players destroy enemies by using common treatments 
such as chemotherapy, radiation, and antibiotics. 
Enemies are also combated by prompting the patient 
to engage in positive self-care behaviors such as 
taking stool softeners to prevent colon tears, 
practicing good mouth care to combat mucositis, 
using relaxation techniques to reduce stress, taking 
their prescribed medicines, or eating for more energy. 
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The design of Re-Mission was based on surveys of 
the views of health professionals about the psycho- 
educational needs of young cancer patients 
undergoing treatment, 13 ' 14 and on reviews of 
literature on the relative efficacy of different psycho- 
education formats used with cancer patients. 1, 2 
Although this background research indicates that a 
videogame such as Re -Mission could well be an 
effective intervention for young cancer patients, 
empirical evidence is needed on how young patients 
view the videogame, especially after they have been 
given the opportunity to use it. An opportunity to 
collect this evidence has been provided by a recent 
multi-site evaluation of the efficacy of Re-Mission, 
which included a rating scale designed to measure 
patients’ perceptions of the credibility and 
acceptability of Re -Mission as a useful intervention. 

Credibility refers to perceived value of the 
videogame as an intervention capable of influencing 
knowledge, attitudes, or health outcomes. 15 
Acceptability refers to ease, enjoyment, and 
harmlessness of use. 16 These constructs were chosen 
to reflect aspects of perception most applicable to the 
type of intervention being evaluated here: a 
videogame about cancer used by young patients 
being treated for cancer. To assess acceptability and 
credibility in this specific study, a 9-item rating scale 
was constructed using items based on those used in 
scales validated in the literature on measurement of 
user’s perceptions regarding concepts such as 
acceptability and credibility of interventions for 
behavioral management, physical or mental 
illnesses. l5 ls In the present study, acceptability is 
measured by 4 items reflecting the extent to which a 
patient would use the videogame as an enjoyable 
activity or recommend it, while credibility is 
measured by 6 items ( 1 item in common) reflecting 
the strength of the patient’s belief that playing the 
videogame could influence a patient’ s knowledge, 
attitudes, or behavior relevant to treatment. Because 
the scale was devised specifically for this study, there 
were at the outset of the study no data available on its 
validity or reliability. The face validity of the scale 
can be assessed by consideration of the items 
themselves (provided in the results section). The lack 
of psychometric information about the scale does 
limit the generality of findings, although some 
psychometric data were collected in the course of the 
study and are reported in the results section. 

Methods 

This report describes part of a larger randomized 
controlled evaluation of the efficacy of Re-Mission, 


relative to a control condition (a commercial 
videogame with no cancer content), as an 
intervention for several health related variables such 
as medication adherence, self care, and quality of life. 
Procedures and measures that were part of the larger 
study, but not relevant to the sub-study reported here, 
are not included in this report. 

Participants 

Approval to conduct the study was obtained from 
relevant research ethics authorities at 34 cancer 
treatment centers in the United States, Canada and 
Australia. Participants (N=375) were recruited from 
those meeting the following inclusion criteria: male 
or female patient 13 to 29 years of age with a cancer 
diagnosis; must be currently receiving treatment and 
is expected to remain on treatment for at least 4-6 
months. Exclusion criteria were: history of seizures 
due to photosensitivity; unable to communicate 
effectively with study personnel in English, Spanish, 
or French; determined by the investigator to be 
incapable of following the study schedule or study 
directions for any reason. Of those recruited, 197 
were randomly assigned to receive access to Re- 
Mission, and 195 actually received the intervention. 
There were 5 participants whose assigned condition 
was crossed over (n=3 treatment to control, n=2 
control to treatment). One hundred seventy six 
patients received access only to an alternative 
videogame. Of the 375 participants enrolled, the data 
from 4 were excluded from analyses due to 
inadequate consent (n=2), ineligibility (n=l) and 
consent withdrawal prior to any data collection (n=l). 
This report is based only on data from those 
participants receiving access to Re-Mission. 

Procedures 

Participants were asked to complete questionnaires 
on three occasions: (Baseline) at the commencement 
of the study; (Follow-up) 1 month after Baseline; and 
(Long Term Follow-up) 3 months after Baseline. The 
only questionnaires relevant to this sub-study were a 
brief demographic questionnaire given at Baseline 
and a rating scale given at Long Term Follow-up 
(LTFU). At Baseline participants were provided with 
a mini-PC loaded with two different videogames that 
could be played using gamepads provided. One 
game was Re-Mission, the other a regular 
commercial adventure game called Indiana Jones and 
The Emperor’s Tomb (Lucas Arts, CA). Participants 
were told they could play both games, but were asked 
to play for at least an hour a week for the next 3 
months. The mini-PC recorded duration of play for 
both games as well as tracking other specifics of Re- 
Mission play, including number of missions played 
and mission completion. At LTFU, 148 of the 
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participants given access to both Re-Mission and 
Indiana Jones completed a 9-item rating scale 
designed to provide a measure of the participants’ 
perceptions of the acceptability and credibility of Re- 
Mission as an intervention ( Items are described in 
Table 1). Participants were asked to respond to each 
item by reading the statement and then circling a 
rating value between 1 and 5 to indicate the extent to 
which they agreed ( 1 = strongly disagree, 5 = 
strongly agree). 

Results 

Data analysis 

Associations between continuous variable were 
analyzed as correlations using Pearson's r. 
Associations between dichotomous or categorical 
variables and continuous variables were analyzed 
using independent-samples /-tests. Associations 
between categorical or dichotomous variables were 
tested as differences between proportions using chi- 
square. The alpha level for rejecting the null 
hypothesis was set at p=. 05. Effect sizes were 
calculated where appropriate as the standardized 
mean difference (Cohen’s d). 

Completers vs. Noncompleters 
Of the 196 participants given access to Re-Mission, 
148 ‘completers’ filled out the 
acceptability/credibility rating scale at Time 3. The 
remaining 48 ‘noncompleters’ did not provide usable 
rating scale data at LTFU for the following reasons : 
completed study but didn’t do the rating scale (16); 
inadequate consent (1); early withdrawals (31). 
Independent-samples /-tests conducted on Re- 
Mission usage data showed that completers played 
Re-Mission longer (M=4.39, SD= 5.16) than 
noncompleters (M=1.21, 371=2.27 )(/( 1 92)= 4.05, 
p<.001). Completers also played more unique game 
‘missions’ than noncompleters (completers, A/=5.20, 
SD = 6.18; noncompleters, M= 2.28, SD= 4.09; 
/(192)=3.00, pc.Ol). Using /-tests for the continuous 
variables and Chi-square test for gender, completers 
and noncompleters were compared to see whether 
they differed significantly on age, gender, prior video 
game use, educational level, or the amount of time 
spent playing Indiana Jones. No significant 
differences were found (all values with p >.05). 

Based on self-reported ethnic identity, most 
participants identified themselves as White (65%), 
Hispanic (25%), or Black (11%). The percentage 
belonging to these ethnic groups who were 
completers was significantly greater for the Hispanic 
group (93%) than for either Black group (55%) or 
White group (74%) (x 2 =11.7,p<.01). 


Responses to the Acceptability/Credibility Rating 
Scale 

The frequencies of each rating given each statement 
in the scale are shown in Table 1. Also shown are the 
frequencies of composite ratings for acceptability and 
credibility calculated by combining ratings for Items 
1, 2, 4, & 7 (acceptability) and Items 2, 3, 5, 6, 8, & 9 
(credibility). Responses to all items were analyzed to 
assess internal consistency of the acceptability and 
credibility subscales using Chronbach's alpha. 19 In 
both instances, internal consistency was good 
(acceptability, alpha = .82; credibility, alpha = .91). 
For both subscales, internal consistency was reduced 
if the common item (Item 2: “I would recommend 
Re-Mission to another young person with cancer”) 
was omitted (acceptability, .72; credibility, .90), 
indicated that the inclusion of this common item was 
not problematic for scale consistency. 

Considering first the composite ratings. Table 1 
shows that the majority of participants rated Re- 
Mission generally to be acceptable and credible as an 
intervention, composite ratings of 4 & 5 being more 
frequent than ratings in the categories 1-3. The mean 
rating for acceptability was 4.1 1 (SD=. 87) and for 
credibility it was 3.72 (SD=.91). Appropriate tests of 
associations between variables (7-test, Pearson's r) 
showed that ratings of acceptability and credibility 
were not significantly associated with participants’ 
gender, age, or prior video game experience (all p 
values >.05). Ratings for the individual items in the 
scale generally followed this same pattern, except for 
item 8: “As a result of this game, I plan to make 
changes in how I manage my cancer treatment”. In 
the case of item 8, twice as many participants gave a 
rating of ‘neutral’ (3) as gave any rating either in 
agreement or disagreement. Those who didn't use 
the ‘neutral’ rating were fairly equally divided 
between agreement and disagreement with the item. 

Playing Time 

Total time (hours) spent playing Re-Mission (not 
including pauses in play) over the 3-month access 
period (playing time) varied widely across 
completers (range 0 - 30.24, M=4.29) and 
noncompleters (range 0 - 9.57, M=. 59). Number of 
unique missions completed also varied widely (range: 
completers, 0-24; noncompleters, 0-23). Distributions 
of playing times for completers and noncompleters 
are shown in Fig. 1. It can be seen that both 
distributions are negatively skewed, with the modal 
playing time for both distributions 0-3 hours. The 
median playing time was 2.74 hours for completers 
and 0.13 hours for noncompleters. 
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Hours spent playing Indiana Jones was similar for 
completers (M= 4.92, SD= 1 4.0 1 ) and noncompleters 
(M= 5.86, SD= 10.66) (t (192) = .80 ns). 

Across all participants with access to both Re- 
Mission and Indiana Jones, total playing times on the 
two games were significantly positively correlated 
(r( 1 94) = .23, p<.00 I ). However, the absolute times 
for the two games are not strictly equivalent, as the 
total times for Indiana Jones include pauses in play, 
whereas the Re-Mission times do not. 

Appropriate statistical tests of association between 
variables (f-test, Pearson's r) showed that playing 
times both for Re-Mission and Indiana Jones were 
not significantly associated with age, gender, or 
previous game experience (all p values >.05). 

Associations Between Playing Time and Rating 
Scale Responses 

Correlational analysis showed a significant 
association between Re-Mission playing time and 
both composite acceptability rating (r (148) =.26, 
[X.O I ) and composite credibility rating (r (148) =.25, 
p<.01). These associations are illustrated in Fig. 2, 
where higher composite acceptability and credibility 
ratings are seen to be associated with longer usage 
times. Indiana Jones playing times were not 
significantly correlated with either acceptability (r 
(148) =.08) or credibility r (148) =.12) ratings. 

The association between playing time and responses 
to individual items on the rating scale was further 
explored using the median-split procedure to classify 
completers either as high users (above median user 
time) or low users (below median user time), then 
comparing the ratings of high and low users on each 
item. Mean ratings on composite acceptability were 
significantly higher for high users (M=4.39, SD=. 73) 
than for low users (M=3.85, SD=. 91) (t (146) =4.00, 
/X.O I ). Mean ratings on composite credibility were 
significantly higher for high users (M= 3.94, SD=.S 1 
than for low users (M=3.50, SD=. 1.06) (t (146) 

=2.90, /?<.() 1 ). Further f-tests showed that differences 
between high and low users’ mean ratings on 
individual items were also significant (/?<.() I ) for all 
items except item 8: “As a result of this game, I plan 
to make changes in how I manage my cancer 
treatment”. A typical distribution of ratings for high 
and low users on most items is shown in Fig. 3, 
which illustrates the distributions for item 7 : 

“Overall, I liked playing Re -Mission”, the item 
showing the largest effect size for this difference 
between means (Cohen's d=. 67). 

As seen in Fig. 3, the distributions of ratings by high 
users, compared with those of low users, were more 


positively skewed toward strong agreement with the 
item statement. That is, high users rated Re-Mission 
more positively than low users on most items. 
However, the distribution of ratings for Item 8, 
shown in Table 1, is unlike the distributions for the 
other items, in that the frequencies are more equally 
distributed across the continuum from 1-5, and the 
modal rating is at the neutral point (3) rather than at 
the positive end of the scale (4 or 5). It is notable 
that the statement being rated in item 8 differs from 
the other items in that it refers to the patient’ s 
perceived control over treatment. 

Discussion 

Most of the participants given access to Re-Mission 
(76%) also completed the acceptability/credibility 
rating scale at LTFU. Because those who did 
complete the scale tended to have played Re-Mission 
more than those who did not complete the scale, the 
question arises as to what extent the responses of 
those completing the scale can be taken to fairly 
represent the whole sample given access to the game. 
However, the range of playing times for the 
completers was larger than for the noncompleters and 
contained both ends of the range for the 
noncompleters, indicating that the difference between 
the mean playing times of the completers and 
noncompleters does not represent a serious restriction 
on generalizations about the whole sample on the 
basis of data from completers only. Also, the analysis 
of demographic variables indicates that there would 
be no bias based on age, gender, prior game 
experience or educational level, but some possibility 
of bias based on ethnicity, to the extent that any 
differences found between ethnic groups on their 
responses to the rating scale might not be validly 
generalized to the whole sample. Accordingly, no 
analysis of differences between ethnic groups was 
undertaken. 

A major finding was that there was wide variation in 
the time spent by participants playing Re-Mission, 
and the amount of time they played was significantly 
associated with their perceptions of Re-Mission as an 
acceptable and credible intervention. Analysis of the 
composite ratings for acceptability and credibility 
indicate that, in general, players considered Re- 
Mission to be a quite acceptable and moderately 
credible intervention. However, those players who 
spent more time playing Re -Mission were generally 
more positive about Re-Mission than those who used 
it less. The simplest alternative interpretations of 
these findings are: (1) More-extensive play with Re- 
Mission results in higher perceived acceptability and 
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credibility; (2) Higher perceived acceptability and 
credibility results in greater interest in playing Re- 
Mission; or (3) Some interaction between both of 
these processes. Of course, acceptability and 
credibility might be related to playing time by 
different processes. For example, acceptability might 
drive playing time, but playing time in turn drives 
credibility. It is possible that both acceptability and 
credibility might change over the course of 
experience with an intervention, as has been shown 
previously. 20 In the case of Re-Mission, it seems 
likely that its perceived credibility, in particular, 
could be affected significantly as continued play 
experience brought a better understanding of what the 
game was about. As measured in this study, 
acceptability and credibility were highly correlated 
(r=.80, p<001), but both might contribute uniquely to 
intervention ‘adherence’, as measured by Re-Mission 
playing time. 

The atypical distribution of ratings for item 8 clearly 
indicates that this item was perceived differently 
from the other items, possibly because it referred to 
how the participant would manage her/his own 
treatment. Perhaps many considered that the 
management of their cancer treatment was not within 
their control, and therefore were not inclined to agree 
with any statement about how they would manage 
their own treatment. 

Turning now to a general consideration of the utility 
of Re-Mission as an adjuvant therapy for young 
cancer patients, it is notable that only 17% of 
completers (and 2.4% of noncompleters) played Re- 
Mission for more than one hour per week on average. 
Also, although the game includes 20 regular 
missions, as well as 3 tutorial missions and 2 two- 
player missions, the mean number of unique missions 
played was only 4.5 (SD= 5.88) and the median 
number was 2 (reflecting the marked skewing of the 
distribution). Perhaps this indicates that play is 
difficult or tiring to maintain, given that many 
patients are struggling with a serious illness and 
debilitating side-effects of treatment. Since longer 
playing time is associated with higher acceptability 
and credibility, it may be valuable to find a way of 
encouraging players to engage with the game for 
longer periods of time. This might be accomplished, 
for example, by making missions shorter and easier 
to perform successfully. However, it is precisely 
because most patients did not play Re-Mission for 
long periods that the generally positive perceptions of 
acceptability and credibility of Re -Mission are so 
encouraging. Also, it may be that the short playing 
times owe less to the characteristics of Re-Mission 


than to a general indisposition to engage in any 
demanding activity by patients who are tired and ill. 

Young cancer patients’ views about the credibility 
and acceptability of alternative concepts of a 
hypothetical game such as Re-Mission were 
previously explored in a study intended to inform the 
feasibility and design of Re -Mission." 1 In that study 
ratings of acceptability or credibility were not found 
to be associated significantly with any of a range of 
demographic or personality variables measured in the 
study. At this time there is little hard evidence on 
which to base attempts to improve the credibility of 
Re-Mission as a cancer intervention. To this end it 
may be useful to conduct interviews with young 
patients who have played Re -Mission, to explore 
with them in more detail the reasons for any 
reservations about both the acceptability and 
credibility of the game. 

The current study indicates that Re-Mission is 
perceived by most young patients as quite acceptable 
and moderately credible, and by a smaller proportion, 
also perceived as an activity worth persevering with 
for at least several hours. It will be important to 
discover to what extent duration of play is associated 
with desired health-related outcomes. In the 
meantime, the current results indicate that there is 
sufficient interest in Re-Mission by young cancer 
patients that it can be regarded as a promising 
addition to the psychoeducation resources available 
to treatment teams. 
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Table 1. Number of Participants Giving Each Rating (1-5) for Each Item on the 
Acceptability/Credibility Scale 


Item 


Rating ( l=strongly disagree, 
5=strongly agree) 



1 

2 

3 

4 

5 








1 

I think it is OK for cancer patients to play Re-Mission 

4 

2 

7 

46 

89 

2 

I would recommend Re-Mission to another young person 
with cancer 

5 

7 

27 

2 

67 

3 

I think Re-Mission would help other people with cancer 

4 

8 

28 

45 

63 

4 

I think that playing Re-Mission would not be harmful to a 
patient 

5 

6 

11 

44 

81 

5 

I believe Re-Mission helped me understand cancer 

8 

14 

21 

53 

52 

6 

I believe I benefited fro using Re-Mission 

10 

14 

37 

41 

46 

7 

Overall, I liked playing Re-Mission 

15 

15 

25 

44 

49 

8 

As a result of this game, I plan to make changes in how I 
manage my cancer treatment 

16 

29 

55 

25 

23 

9 

As a result of playing this game, I am more likely to take 
my medicine the way my doctor prescribed it 

14 

15 

37 

30 

52 








Acceptability 

(composite) 

3 

4 

28 

58 

55 

Credibility 

(composite) 

3 

19 

35 

55 

36 
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Re-Mission total playing time (hours) 


Fig. 1. The distribution of total playing times is shown for all participants who completed (gray) or did not 
complete (white) the acceptability/credibility rating scale. 
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Acceptability rating 



Credibility rating 


Fig. 2. Regressions on log transformed total playing time (log hours user time) are shown for composite 
ratings of acceptability (upper panel) and credibility (lower panel). 
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Frequency Frequency Frequency 
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Beale et al. 



Composite acceptability rating (1=strongly disagree, 5 = strongly agree) 



Composite credibility rating (1=strongly disagree, 5 = strongly agree) 



Rating (1=strongly disagree, 5 = strongly agree) 


Fig. 3. Distributions of ratings for composite acceptability, composite credibility and Item 7 are shown for 
high users (white) and low users (gray). 
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